Player aggregation in the traveling inspector model

We consider a model of dynamic inspection/surveillance of a number of facilities in different geographical locations. The inspector in this process travels from one facility to another and performs an inspection at each facility he visits. His aim is to devise an inspection/ travel schedule which minimizes the losses to society (or to his employer) resulting both from undetected violations of the regulations and from the costs of the policing operation. This model is formulated as a non-cooperative, single-controller, stochastic game. The existence of stationary Nash equilibria is established as a consequence of aggregating all the inspectees into a single “aggregated inspectee”. It is shown that such player aggregation causes no loss of generality under very mild assumptions. A notion of an “optimal Nash equilibrium” for the inspector is introduced and proven to be well-defined in this context. The issue of the inspector’s power to “enforce” such an equilibrium is also discussed

Hamiltonian cycles and subsets of discounted occupational measures

We study a certain polytope arising from embedding the Hamiltonian cycle problem in a discounted Markov decision process. The Hamiltonian cycle problem can be reduced to finding particular extreme points of a certain polytope associated with the input graph. This polytope is a subset of the space of discounted occupational measures. We characterize the feasible bases of the polytope for a general input graph $G$, and determine the expected numbers of different types of feasible bases when the underlying graph is random. We utilize these results to demonstrate that augmenting certain additional constraints to reduce the polyhedral domain can eliminate a large number of feasible bases that do not correspond to Hamiltonian cycles. Finally, we develop a random walk algorithm on the feasible bases of the reduced polytope and present some numerical results. We conclude with a conjecture on the feasible bases of the reduced polytope.Comment: revised based on referees comment

On production and abatement time scales in sustainable development. Can we loosen the sustainability screw ?

In this paper we carry out a preliminary exploration of a time scales' conjecture, which postulates that "reasonable" notions of sustainability must include a suitable synchronisation of time scales of both the processes of human development and those of the natural environment. We perform our analysis within a coarse, ?ve variable, model of man-nature interactions expressed as a system of differential equations where production and human capital are coupled with both renewable and non-renewable natural resource. We demonstrate a phenomenon that we name the "sustainability screw" that describes a spiral like trajectory of the three key variables: non-renewable and renewable resources as well as the production capital. Under many plausible scenarios, this spiral tends unacceptably fast to an undesirable equilibrium. However, we also show that by adjusting the ratio of "intensity of production effort" and "intensity of abatement effort", parameters of the relative time scales of production and natural recovery processes can be altered in a manner that produces, arguably, more sustainable trajectories.sustainable optimization systems, viability, multiple time scale

Weighted Markov Decision Processes with perturbation

In this paper we consider the weighted reward MDP’s with perturbation. We give the proof of existence of a delta-optimal simple ultimately deterministic policy under the assumption of “scalar value”. We also prove that there exists a delta-i-optimal simple ultimately deterministic policy in the perturbed weighted MDP, for all e E [0, e*) even without the assumption of “scalar value”

Perturbation theory for semi-Markov control problems

In earlier work, the authors considered the perturbation of systems undergoing Markov processes in which the times between two consecutive decision time points were equidistant. They now consider perturbations of processes for which the times between transition are random variables. These are called semi-Markov processes

The embedding of the traveling salesman problem in a Markov Decision Process

In this paper we derive a new LP-relaxation of the Traveling Salesman Problem (TSP, for short). This formulation comes from first embedding the TSP in a Markov Decision Process (MDP: for short), and from perturbing this MDP appropriately

Perturbation and stability theory for Markov control problems

A unified approach to the asymptotic analysis of a Markov decision process disturbed by an ε-additive perturbation is proposed. Irrespective of whether the perturbation is regular or singular, the underlying control problem that needs to be understood is the limit Markov control problem. The properties of this problem are the subject of this study

Aggregation-disaggregation algorithm for e2-singularly perturbed limiting average Markov control problems

In this paper we consider a singular perturbation of order 2 for a Markov decision process with the limiting average reward criterion. We define a singular perturbation of order 2 in the following sense: we assume that the underlying process is composed of n separate irreducible processes, and that a small e-perturbation is such that it ”unites” these processes into m separate irreducible processes. Then another small e2-perturbation is such that it “unites” these latter processes into a single irreducible process. The present paper is organized as follows: In Section 2, we formulate the singular perturbation of order 2. In Section 3, we give explicitly the limit Markov Control Problem (limit MCP), that is entirely different from the original unperturbed MDP, which forms an appropriate asymptotic approximation to a whole family of perturbed problems. Thus only the single limit MCP needs to be solved